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SUMMARY 


In  the  classical  Secretary  Problem,  the  player  tries  to 
choose  the  best  object  of  a  sequentially  ordered  set  of  size  N  . 

The  value  of  each  object  is  given  by  its  rank  only.  At  any  stage, 
the  player  knows  the  rank  of  the  current  object  relative  to  those 
already  seen.  Once  rejected,  an  object  cannot  be  chosen  later. 

In  this  paper,  a  generalized  Secretary  Problem  is  discussed. 
The  player  is  given  d  choices  to  choose  all  of  the  d  best 
objects.  The  optimal  procedure  is  found  by  converting  the  d 

*•  Ck* 

choice  Secretary  Problem  into  a  "walk"  in  a  two-dimensional  grid. 

A  simple  approximation  to  the  optimal  strategy  rule  is  also 
presented . 


1.  INTRODUCTION 


The  classical  Secretary  Problem  beqins  with  the  premise 
that  N  objects  or  individuals  can  be  ranked  from  best  (rank  =  1) 
to  worst  (rank  *  N) .  As  each  object  is  shown  to  the  experimentor 
(or  player),  he  is  able  to  rank  it  only  in  relation  to  those 
objects  that  he  has  already  seen.  The  player  may  select  the  current 
object,  at  which  point  the  procedure  ends,  or  he  may  elect  to  reject 
the  current  object  and  sample  the  next.  The  player  also  operates 
under  the  restriction  that  once  an  object  has  been  rejected,  it  may 
not  be  chosen  later.  I f  no  choice  has  been  made,  he  is  required  to 
choose  the  Nth  object. 

The  optimal  procedure  for  the  player  to  follow  depends  upon 
the  type  of  reward  he  receives.  Oftentimes,  the  player  tries  to 
pick  the  best  object.  In  this  case,  the  optimal  procedure  maximizes 
the  probability  of  success.  This  problem  has  been  solved  in  a 
variety  of  ways,  by  Bindley  (1961),  Dynkin  (1963),  Gilbert  and 
Mosteller  (1966),  Dynkin  and  Yushkovich  (1969),  DeGroot  (1970), 

Chow,  Robbins,  and  Siegmund  (1971),  and  Sirjaev  (1973).  The  problem 
also  appeared  in  Marvin  Gardeners'  Scientific  Amer ican  column  (Fox 
and  Marnie  (1960),  Moser  and  Pounder  (I960)), and  in  the  American 
Mathemat  ical  Monthly  "Problems  and  Solutions"  (Bissinqer  and  Siegel 
(1963),  Bosch  (1964)). 

Several  authors  have  discussed  the  Secretary  Problem  when 
the  player  is  given  one  choice  to  choose  any  of  the  d  best  objects. 
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Gilbert  and  Mosteller  (1966)  discussed  the  case  d  «=  2.  Gusein- 
Zade  (1966),  Dynkin  and  Yushkevich  (1969)  and  Rasmussen  (1974  and 
1975)  have  generalized  the  problem  further. 

Another  related  problem  that  is  of  interest  is  to  allow 
the  player  more  than  one  choice  to  select  the  best  object.  This 
case  has  been  discussed  by  Gilbert  and  Mosteller  (1966). 

The  problem  that  will  be  discussed  in  this  paper  is  a 
combination  of  the  two  generalizations  given  above.  The  player  is 
given  d  choices  to  select  all  of  the  d  best  objects.  We  will 
derive  the  optimal  procedure,  which  in  this  case  maximizes  the 
probability  of  selecting  the  d  best.  The  form  of  this  optimal 
procedure  is: 

Procedure  A:  Choose  d  starting  times  t^,  ...,  t^  . 

Do  not  select  any  of  the  first  tj-1  objects.  The  d  choices  are 
then  made  in  one  of  three  ways: 

(1)  At  any  time,  select  an  object  that  is  better  than  at 
least  one  of  the  objects  already  chosen. 

(2)  Make  the  j*"*1  choice  of  the  kt^1  object  if  k  a  t.  , 

and  the  k*^  object  has  rank  j  or  better,  where  1  s  j  &  d. 

(3)  If  m  choices  have  been  made,  0  £  m  <  d  ,  and  d-m 

objects  are  left,  all  of  the  remaining  objects  must  be  chosen. 

The  procedure  ends  when  the  dth  choice  is  made. 

The  probability  of  success  using  Procedure  A  will  be  denoted 

by  P^(CS;  d , t ^ , . . . , t^ , N )  ,  where  CS  means  correct  selection.  This 

will  be  abbreviated  by  PA(CS)  when  there  is  no  ambiguity.  The 
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optimal  starting  times  t^. 


•  •  •  $ 


t*,  are  chosen  to  maximize  PA<CS) 
for  a  given  N  .  As  in  Gilbert  and  Mosteller  (1966),  an  object 
that  is  better  than  all  others  seen  so  far  will  be  called  a  candidate. 

In  addition,  we  will  consider  an  approximation  to  the  optimal 
rule,  based  on  the  optimal  starting  time  for  the  last  choice,  t*  . 

U 

The  approximately  optimal  rule  has  the  same  form  as  Procedure  A, 
but  the  starting  times  are  easier  to  compute.  These  approximately 
optimal  starting  times  do  not  necessarily  maximize  P^fCS). 


2.  THE  d  CHOICE  PROBLEM  -  OPTIMAL  PROCEDURE 


In  this  section,  we  will  show  that  the  optimal  procedure 
for  the  d  choice  Secretary  Problem  has  the  form  of  Procedure  A, 
with  starting  times  t|,  ...»  t£  .  If  the  player  is  forced  to 
choose  the  last  d-m  objects,  as  in  (3)  of  Procedure  A,  he  may 
choose  an  object  that  is  worse  than  one  already  rejected.  If  such 
an  object  is  chosen,  he  cannot  succeed  in  choosinq  the  d  best. 
Unless  the  player  makes  choices  usinq  (1)  or  (2),  he  will  not 
succeed.  Thus,  without  affectinq  P  (CS)  ,  the  type  (3)  choices 
can  be  ignored,  since  they  are  included  only  to  insure  that  d 
choices  are  made.  When  computinq  P^(CS),  it  will  be  assumed  that 
this  has  been  done,  and  that  choices  are  made  usinq  only  (1)  and 
(2);  Eliminatinq  tvpe  (3)  choices  will  make  certain  aspects  f  the 
construction  of  the  optimal  procedure  clearer. 

Nikolaev  (1977)  has  shown  that  Procedure  A  is  optimal  for 

the  case  d=2.  In  addition,  both  he  and  Glasser  (1978)  have  derived 

exact  expressions  for  P^fCS:  2,tj,t2,N).  An  exact  expression  for 

P.  (CS;  3 , t , , t_ , t , , N)  has  also  been  derived  by  Glasser.  The  task 
A  1  l  i 

of  deriving  an  exact  expression  for  P^fCS)  ,  however,  is  quite 
tedious  even  for  d=3. 

Theorem  2.1  will  show  that  Procedure  A  is  optimal,  and  give 
a  method  to  compute  the  optimal  startinq  times  t^,...,t*  .  In  the 
next  section,  we  will  give  a  specific  formulation  for  t^  ,  and 
use  it  to  construct  an  approximation  to  the  optimal  rule. 


4 


=  ' 

To  begin  with,  define  f  =  N-d  ,  and  consider  the  two- 
dimensional  grid  of  points  (i,j),  where  i  »  0,1,..., d  ,  and 
j  *  0,1,..., f.  When  the  player  is  at  the  point  (i,j),  it  means 
that  of  (i  +  j)  items  sampled  so  far,  he  has  accepted  i  and  rejected 
j  of  them.  From  the  point  (i,j),  he  will  move  to  the  point  (i+l,j) 
if  he  accepts  the  next  object,  and  (i,j+l)  if  he  rejects  it. 

Any  rule  for  selecting  the  d  best  objects  can  be  put 

into  a  1-1  correspondence  with  a  speci f ic"wa lk"  on  the  grid.  As 

an  example,  consider  Procedure  A.  If  the  player  is  at  the  point 

s  t 

(i,;j),  and  the  (i  +  j-fl)  object  is  ranked  i  or  better,  he  will 

s  t 

accept  it  and  move  to  (i+l,j).  If  the  (i+j+1)  object  has 

rank  i+1,  and  i+j+1  i  tj+j  *  then  the  player  also  accepts  the 
object  and  moves  to  (i+l,j).  Otherwise,  the  object  is  rejected 
and  the  player  moves  to  (i,j+l). 

Any  time  the  player  "walks"  off  the  grid,  he  loses  the 
game.  For  instance,  at  the  point  (d+1,3)  ,  he  has  chosen  too 
many  objects.  At  the  point  (3,f+l),  there  are  not  enough  objects 
left  to  make  d  choices. 

It  will  be  seen  in  Theorem  2.1  that  the  optimal  procedure 
requires  the  player  to  choose  an  object  better  than  any  object 
already  selected,  and  reject  an  object  worse  than  any  object 
already  selected.  Thus,  starting  from  the  point  (0,0),  the  player 
succeeds  if  and  only  if  he  reaches  the  point  (d,f).  If  at  any  time 
the  player  chooses  an  object  that  is  not  among  the  d  best,  the 
optimal  procedure  will  force  him  to  choose  too  many  objects.  If 
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the  player  rejects  an  object  that  is  one  of  the  d  best,  the 
optimal  procedure  will  not  allow  him  to  make  d  selections. 

A  utility  function  can  be  defined  that  reflects  the  player's 
desire  to  choose  the  d  best  object.  This  utility  is  a  function 
of  the  ranks  of  the  objects  chosen,  and  is  given  by: 


/I  ( i i , i 2  » • . • # i^ )  "  (1,2, ...,d) 

0(i1,i2,...,id)  =  J  (2.1) 

*  0  otherwise . 


The  player  is  interested  in  choosing  the  d  best,  and  is  not 
concerned  with  the  order  of  their  arrival.  Therefore,  the  ranks  of 
the  objects  chosen  can  be  permuted  in  any  order  without  affecting 
the  value  of  the  utility  function. 

Since  the  player  succeeds  in  choosing  the  d  best  if  and 
only  if  he  reaches  the  point  (d,f),  the  utility  function  in  (2.1) 
can  be  related  to  the  qrid  described  earlier.  Assign  each  point 
on  the  qrid  a  conditional  expected  utility  u(i,j).  The  values 
u(i,j)  are  conditioned  by  the  player  havino  accepted  i  out  of 
i+j  objects  seen  so  far.  It  is  also  assumed  that  at  each  step, 
the  player  will  elect  to  continue  in  an  optimal  manner.  The 
computation  of  these  u(i,j)'s  is  the  subject  of  the  next  theorem. 
Corollary  2.1  shows  that  Procedure  A  is  the  optimal  procedure  for 
the  d  choice  Secretary  Problem. 
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Theorem  2.1: 


u  ( i ,  j  )  =  i • u  ( i  + 1 ,  j )  +  j‘u(i,  j  + 1 )  ^max[u(i,  j  +  1 )  ,  u  ( i  +  1 ,  j  )1  ( 2  2) 


where  i  =  0,1, ...(d  and  j  =  0,1, ...,f  .  Initial  conditions 
are  given  by: 


u  (d,  f ) 


(2.3) 


u (d+ 1 ,  j )  =  u(i,f+l)  =  0  , 


for  i  =  0,l,...,d-l,  and  j  =  0,l,...,f-l  . 

s  t 

Proof :  The  rank  of  the  (i+j+1)  object  can  be  any  of 

1, 2, . . . , i+ j+1,  with  equal  probability.  This  object  will  be  better 
than  at  least  one  of  the  objects  already  selected  if  it  has 
relative  rank  i  or  better,  with  probability  i/(i+j+l).  When  the 
object  is  better  than  an  object  already  selected,  it  must  be 
selected  if  the  player  wishes  to  continue  the  possibility  of 
selecting  the  d  best. 

Otherwise,  if  the  player  rejects  an  object  ranked  i  or 

s  t 

better,  one  of  two  possibilities  can  occur.  If  the  (i+j+1) 
object  is  not  one  of  the  d  best,  he  has  already  chosen  an  object 

C  f 

that  cannot  be  one  of  the  d  best.  If  the  (i+j+1)  object  is 
one  of  the  d  best,  the  player  will  have  rejected  an  object  that 
is  one  of  the  d  best.  In  either  case,  the  player  cannot  choose 
all  of  the  d  best. 
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Thus,  an  object  ranked  between  1  and  i  must  be  accepted. 

In  this  case,  the  player  moves  from  (i,j)  to  (i,j+l). 

s  t 

When  the  (i+j+1)  object  is  worse  than  at  least  one  of 
the  objects  already  selected,  with  relative  rank  between  i+2  and 
i+j+1,  it  is  better  to  reject  it.  The  reasons  are  similar  to 
those  given  above.  The  probability  that  the  rank  is  i+2  or 
greater  is  j/(i+j+l).  The  player  in  this  case  moves  from  (i,j) 
to  (i , j  +  1 ) . 

st 

A  question  arises  then,  only  in  the  case  that  the  (i+j+1) 
object  is  neither  better  than  any  objects  already  selected,  nor 
worse  than  any  already  rejected.  This  means  that  the  rank  of  the 
new  object  is  i+1  ,  and  this  occurs  with  probability  l/(i+j+l). 

For  this  last  case,  the  best  course  of  action  depends  on 
the  expected  utility  if  the  object  is  rejected,  u(i,j+l),  and 

the  expected  utility  if  the  object  is  selected  u(i+l,j).  Clearly, 

if  u(i+l,j)  >  u(i,j+l)  it  is  better  to  accept  the  new  object. 

When  u(i+l,j)  <  u(i,j+l)  it  is  better  to  reject  the  new  object. 
When  the  utilities  are  equal,  the  choices  are  equivalent. 

So  the  conditional  expected  utility  at  (i,j)  is  given 
by  (2.2).  Usinq  (2.2),  with  initial  conditions  (2.3),  the  values 
of  u(i,j)  can  be  computed  backwards  beqinninq  with  u(d,f)  *  1. 

The  optimal  procedure,  following  the  reasoning  of  the 
above  paragraphs,  is  given  by: 

s  t 

Procedure  A' :  Accept  the  (i+j+1)  object  if  it  is  better 
than  at  least  one  of  the  objects  already  selected.  Reject  the 
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s  t 

(i+j+1)  object  if  it  is  worse  than  at  least  one  object  already 
rejected.  Otherwise,  accept  or  reject  the  (i+j+l)st  object  if 
u(i+l,j)  is  greater  than  or  less  than  u(i,j+l)  respectively. 

As  an  example,  assume  that  the  player  is  to  pick  the  best 
two  out  of  seven  objects.  The  grid  for  the  optimal  procedure  is 
given  below,  with  N  =  7,  d  =  2,  f  =  5: 


21 

4m 

7 

i 

t 

k 

13 

9 

5 

105 

n 

t 

k 

7 

86 

47 

10 

4To  t 

r 

_J55 _ 

-1 

rr 

34  3 

171 

30 

T?i5o 

k 

Uo 

T05 

745 

402 

60 

2525  „ 

r260 

420 

745 

4f>2 

60 

2520 

25T0 

< 

TTST) 

0  1  2 


The  arrows  connect  the  path  with  the  maximal  expected  utilities. 

It  will  be  shown  in  the  next  corollary  that  Procedure  A 
with  starting  times  tf,...,t*  is  the  same  as  Procedure  A'.  The 
following  definition  will  be  helpful.  Consider  the  diagonal  set  of 
points  (0,m),  (l,m-l),  (2,m-2),  ...,  (m-1,1),  (m,0).  Call  the 
set  of  points  on  this  diagonal  that  are  also  on  the  grid  an  axis 
of  constant  sample  size  m  .  When  no  ambiguity  exists,  the  term 
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will  be  abbreviated  as  an  axis  of  size  m  ,  or  simply  called  an 
axis.  In  the  above  example,  the  points  (0,5),  (1,4),  and  (2,3) 

are  an  axis  of  size  5. 

Note  that  in  the  above  example,  there  is  one  maximal  utility 
on  each  axis,  and  the  utilities  decrease  for  points  further  away 
from  the  optimal  path.  This  is  the  subject  of  the  following 
corol lary . 

Corollary  2.1:  On  each  axis  of  size  m  ,  m  =  0,1,...,N-1, 
u(i,j)  is  a  unimodal  function  of  i  ,  and  is  monotonically 
decreasing  on  each  side  of  the  mode. 

Proof :  By  backward  induction.  Assume  that  d  <  f  .  By 

(2.2),  u(d-l,f)  =  d/N  <  f/N  *  u(d,f-l)  .  Thus,  the  corollary  is 
true  for  m  =  N-l  .  Assume  that  the  corollary  is  true  for  m+1, 
and  let  i  +  j  =  m  .  Consider  the  points  ( i  —  1 ,  j  +  2) ,  (i,j  +  l)  ,  and 
(i+l,j),  and  assume  that  they  lie  above  the  optimal  path.  Then 
the  conditional  expected  utilities  for  these  points  satisfy 
u  ( i  - 1 ,  j  +  2 )  =  a  <  u  ( i , j  + 1 )  =  b  <  u  { i  + 1 , j )  =  c  . 

From  (2.2),  the  following  may  be  computed: 


.  ,  .  .  .  ,  ,  ib  ( j  +  1 ) a 

u(i-i,  3+1)  =  t+yH —  ' 

u  ( i  ,  j ) 


( i+1 ) c  +  ]b 
i  +  i  +  1 


Since  a  <  b  <  c,  u(i,j)  -  u(i-l,  j  +  1)  >  0  . 

In  a  similar  manner,  it  can  be  shown  that  for  points  below 
the  optimal  path,  as  the  first  argument  increases  the  expected  utility 
decreases.  This  completes  the  proof. 
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Corollary  2.1  shows  that  above  the  optimal  path,  the  player 
will  select  an  object  when  given  a  choice.  Below  the  optimal  path, 
he  will  reject  an  object  when  given  a  choice.  Using  the  optimal 
strategy  A’,  the  player  will  attempt  to  stay  as  close  as  possible 
to  the  optimal  path. 

In  other  words,  if  the  j1"*1  choice  has  not  been  made,  and 
the  kf^  object  is  ranked  j*^,  where  kit*,  the  optimal 
strategy  A'  says  to  choose  the  object.  Otherwise  select  objects 
better  than  at  least  one  object  already  selected,  and  reject  any 
object  that  is  worse  than  any  object  already  rejected.  This  is  the 

same  as  Procedure  A  with  starting  times  t*,...,t*  . 

1  d 

In  the  example  computed  before,  the  optimal  starting  times 
are  given  by  t*  =  2,  t*  =  5.  The  example  is  also  an  application 
of  c orol 1  ary  2.1. 

Using  the  utility  function  (2.11,  it  can  be  seen  that  the 
probability  of  success  given  that  the  player  is  at  the  point  (d,f) 
is  the  same  as  the  utility  u(d,f)  =  1  .  Working  backwards,  the 
conditional  expected  utility  u(d-l,f)  is  the  same  as  the 
conditional  probability  that  the  player  wins,  given  that  he  is  at 
the  point  (d-l,f)  and  that  he  will  continue  to  the  last  step  in 
an  optimal  manner.  Continuing  to  work  backwards,  the  expected 
utility  u(0,0)  is  the  same  as  the  probability  that  the  player 
wins  using  the  optimal  starting  times  t*,...,t*,  and  making  the 
optimal  choice  at  each  step: 

u (0,0)  =  PA<CS;  d,t*,...,t£,N)  . 
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In  the  above  example,  this  probability  is  given  by  u(0,0)  = 

745/2520  =>  .295635. 

Tables  1  and  2  contain  the  optimal  starting  times  and  the 
probability  of  success  for  the  d  choice  Secretary  Problem  for 
d  -  2,5  and  N  =*  d(l)  50(10)  100,  1000.  All  of  these  calculations 
were  done  using  Theorem  2.1.  As  expected,  when  d  *  2,  the  results 
are  the  same  as  those  derived  using  the  exact  expression  of 
Nikolaev  (1977)  or  C.lasser  ( 1978  )  . 


In  Tables  l  and 

2,  it 

appears 

that 

the  optimal  probability 

of  success  is  a 

non- increasing 

function  of 

N  .  This  is  the 

subject  of  the 

next  theorem  and  corollary. 

Theorem 

2.2: 

If  d  < 

N  , 

VCS; 

d ,  t  j  , .  .  . 

,td,N) 

1  N 

3.  ±  ' 

N  i-1 

r,  (CS;  d,t.(l)  ,  .  .  .  ,t  i1  ’  , N- 1 )  , 

Ala 

where 

t(i)  - 
3 

t . 

) 

if 

i  ,  tj 

M 

V1 

if 

i  <  t  . 
1 

Proof  : 

Among 

N  applicants, 

the 

probability  that  the 

worst 

is  in  position  i 

is  1 /N ,  so 

that 

PA (CS;  d , t 1 , . . . , td,N)  * 

N 

p 

A 

l*  1 

(CS;  d,tL 

t  •  •  •  9  ^  » 

N  |  i )  , 

where 

PA(CS:  d,tj , . . . ,td,N| i)  is 

the  conditional 

probabiltiy  of 

success 

for 

procedure  A  given  that 

the  worst  (rank  *  N)  object  is  sampled  i 


,  th 
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Consider  two  cases: 


Case  (a) :  >  1  (so  the  first  object  is  not  chosen) . 

For  any  value  of  i  ,  the  worst  object  will  not  be  chosen,  since 
it  is  worse  than  an  object  already  rejected  (the  first).  Thus, 
in  this  case,  the  itl1  object  could  be  ignored  without  affecting 
the  probability  of  success,  provided  we  renumber  properly,  i.e., 
PA(CS;  d.tj, .. . ,td,N|i)  *  pA(csf  d,tjl) . tdx),  N- 1 )  . 

Case  (b) :  t^  =  l(so  the  first  object  is  chosen).  The  same 
argument  shows  that  P^fCS;  d , t ^ , . . . , t^ , N | i )  =PA(CS;  d  ,  t| 1 ^ , . . . , t^* ^ 
provided  i  is  not  too  small,  namely  if  i>d  or  t^>i  •  Otherwise 
i  k  d  and  t..  =  j  for  all  j  k  i  ,  in  which  case  all  of  the  first 
i  objects  are  chosen,  including  the  worst,  so 

PA(CS;  d.tj,  ..  .  ,td,N)  =  0  s.  PA(CS;  d  ,  t  ^  1  ]  ,  .  .  .  ,  tj 1  ]  ,  N-l )  . 

Corollary  2.2:  P^CS;  d , t *,..., t* , N)  i  PA(CS;  d  ,  t*  ,  .  .  . , t* ,N 

Proof :  The  corollary  follows  directly  from  Theorem  2.2. 

We  have  seen  that  PA(CS;  d , t ?,..., t* ,d  + 1 )  *  Pft (CS ; t J , . . . , t* 
PA(CS;  t t^ , N )  *  ...  We  shall  now  compute  a  lower  bound  for 

the  probability  of  success.  Consider  the  strategy  which  rejects  the 
first  r  and  picks  when  possible  anything  after  that.  One  reaches 
(0,r)  with  probability  1  and  beyond  that  the  probabilities  at  (i,j) 
are  ( i+1 ) / (i+ j+1 )  to  (i+l,j)  and  j/(i+j+l)  to  (i,j+l).  Any  path 


N-l)  , 


1)  . 


d  +  2) 
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There  is  an  obvious  symmetry  between  the  optimal  strategies 
for  choosing  the  best  d  out  of  N  and  the  best  N-d  out  of  N  . 

In  particular,  for  N=2d,  we  have  u(i,j)  =  u(j,i)  and  the 
maximum  on  each  diagonal  i  +  j  =  r  occurs  at  (r/2,  r/2)  for  even 
r  and  ((r+l)/2,  (r-l)/2)  and  ((r-l)/2,  (r+l)/2)  for  odd  r  . 

In  other  words,  choose,  if  possible,  when  i  <  j  ,  reject  if  possible, 
when  i  >  j  .  When  i  =  j,  either  decision  will  do. 

Knowing  the  optimal  strateqy  for  the  case  N=2d,  the 
probability  of  success  can  be  calculated  by  starting  at  (0,0). 

Let  p(i,j)  be  the  probability  of  getting  from  (0,0)  to  (i,j). 

Since  one  can  only  qet  to  (i,j)  from  either  ( i —  1 , j )  or  (i,j-l), 
we  have : 

p(i,j)  *  pc(i-l,j)  p(i-l,j)  +  Pr ( i , j - 1 )  p(i,j-l)  ,  (2.4 
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where  pc  ( i , j )  is  the  probability  of  choosing  an  object  when  at 
(i,j)  and  pr<i,j)  is  the  probability  of  rejecting  an  object 
when  at  ( i  ,  ] )  . 

Pc  ( i »  j )  -  (i  +  1) / (i  + j  +  1 )  if  i  <  j 

-  i/  (i  +  j  +  1)  i  >  j 

1 


Pr  C i , j )  =  j  (i  +  j  +  1 ) 


if  i  <  j 


=  (j+l)/(i+j+l)  i  >  j 

1 

=  5  x  =  j. 


(Do  the  cases  i=j,  i»j+l,  i>j+l  separately;  the  rest  follow  by 
symmetry.)  In  particular,  p(d,d)  =  l/(d+l)  »  PA(CS;  d,t*,...,t 
The  case  N  *  2d+l  can  be  handled  similarly  once  the 
strategy  is  determined.  Using  an  inductive  proof  similar  to  that 
of  corollary  2.1,  it  can  be  shown  that  the  maximum  occurs  at 


IS 


(r/2,r/2)  or  (  (r-l)/2, (r+l)/2)  depending  on  whether  r  is  even 
or  odd.  (More  speci f ical ly ,  it  can  be  shown  that  for  eveh  r  , 
u(r/2, r/2)  >  u  ( (r-2>/2, (r+2) /2)  >  u ( (r+2) /2, (r-2) /2)  >  ...  > 
u( <r-2i)/2,  (r+2i)/2)  >  u ( <r+2i ) /2 , (r-2i ) /2)  and  for  odd  r  , 
u ( (r-l)/2, <r+l)/2)  >  u(  (r+l)/2, (r-l)/2)  >  ...  >  u  (  (r-2i-l)/2, 

(r-2i ♦  1 ) /2)  >  u((r-2i+i)/2,  (r-2i-l)/2)  >...). 

The  strategy,  therefore,  is  to  choose,  if  possible  when 

i  <  j,  and  reject  if  possible  when  i  2  j.  Knowing  this,  we  may 

proceed  as  in  the  case  N  =  2d .  For  this  case,  the  values  of 

p  (i,j)  and  p  (i,j)  are  given  by: 
c  x* 

p  ( i ,  j )  ■=  (i+l)/(i+j+l)  if  i<j 

=>  i/ (i  + j  +  1 )  i  2  j . 


Pr<i,j)  a  j/(i+j+l)  if  i  <  j 

=  ( j+l)/(i+j+l)  i  2  j. 

(0,0)  and  using  (2.4),  we  can  show  by  induction 

=  1/j  if  i  <  j 

»  j/i  (i  + 1 )  i  2  j . 

(Do  the  cases  i  <  j-1,  i  *  j  —  1 ,  i  =  j  and  i  >  j  separately.) 

Thus,  P.(CS:  d , t* , . . . , t* , 2d  + 1 )  =  l/(d  +  l).  Note  the 
A  Id 

coincidence  that  P  (CS;  d , t* , . . . , t£ , 2d )  =  P^(CS;  t| , . . . , t^ , 2d+ 1 ) . 
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Starting  with 
that 

p(x, j) 


I 


Lemma  3.1:  (1)  If  b(i+l,j)  <  c(i,j+l),  then 

b  ( i  ,  j )  =  b(i+l,j)  +  b(i,j+l)  .  (3.2) 

(2)  If  b(i+l,j)  >  c(i, j+1)  ,  then, 

c(i,j)  =  c(i+l,j)  ♦  c(i,j+l)  .  (3.3) 


Proof:  First,  to  show  both  (1)  and  (2),  it  is  necessary  to 

show  that  b(i+l,j)  <  or  >  c(i,j  +  l)  if  and  only  if  u(i+l,j)  < 

or  > u (i, j  +  1) .  By  (3.1),  these  assertions  must  be  true,  since: 


b(i  +  l,j)  -  c  ( i , j  +  1 ) 


To  show  (1),  if  b(i+l,j)  <  c(i,j+l)  ,  then  u(i+l,j)  < 
u(i,j+l).  From  (3.2),  this  means  that: 


u (i , j  ) 


rnn  u(i+i,j)  *  u(i, j+D 


(3.4) 


Multiplying  both  sides  of  (3.2)  by  (^)/i(x^)  gives  the  result. 
(2)  is  proved  in  a  similar  manner. 

Lemma  3.1  says  that  below  the  optimal  path,  the  b(i,j)'s 
can  be  found  using  (3.2),  while  on  or  above  the  optimal  path  the 
c(i,j)'s  can  be  found  using  (3.3).  Using  Lemma  3.1,  an  expression 
for  the  last  starting  time  is  derived  in  the  next  theorem. 

Theorem  3.1:  The  last  optimal  starting  time  is  given  by: 


fcd 


minim. 


1 


(3.5) 


18 


Proof:  Assume  that  f  >  d  . 


From  (2.2),  it  can  be  seen 


that : 

u(i.f)  -  (l*f)  /  (“)  . 

for  i  ■  0,l,...,d-l  .  Thus,  from  (3.1), 

c(i,f)  =  f  i  =  0, 1 , . . . ,d-l . 

Aqain  usinq  (2.2),  the  conditional  expected  utilities  of  the  last 
column  are  given  by: 

u(d,  j)  =  (d^j)  /(J)  ' 

for  j  =  0,1, ... ,f-l  •  So  from  (3.1),  the  last  column  of  c(i,j)'s 
is  given  by: 

c(d,j)  =  1/j  j  *  1 , 2 , . . . , f- 1 . 

Using  (3.1)  and  (3.6),  it  can  be  seen  that: 

b(d,j)  =  1/d  j  =  0,1,  .  . . ,f-l. 

Now  consider  the  values  of  c(i,j)  and  b ( i , j )  for  the 
next  to  last  column.  Start  with  the  point  (d— l,f— 1)  and  decrease 
the  second  argument  one  step  at  a  time.  As  long  as  c(d-l,j)  < 
b(d,j-l)  =  1/d  Lemma  3.1  gives: 

f  ! 

c(d-a,j)  =  L  r-  . 

k-j  k 


(3.6) 
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20 


21 


-■> 

It  can  be  seen  from  corollary  3.1  that  if  N  increases  to 
infinity  while  d  remains  constant,  the  following  is  true: 

lim  t*/N  *  e  . 

N  ►<*>  ° 

The  importance  of  this  section  is  shown  in  the  next  section. 
There,  a  rule  will  be  given  to  approximate  the  optimal  starting 
times.  This  rule  will  allow  the  player  to  calculate  approximately 
optimal  starting  times  much  more  readily  than  the  true  optimal 
starting  times. 
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4.  THE  LAST  CHOICE  RULE 

Even  for  d  =  2,  computing  the  qrid  in  Section  2  is  more 
direct  and  is  a  faster  way  to  find  the  optimal  starting  times  than 
the  method  outlined  in  Nikolaev  (1977)  or  Glasser ( 1 97 8 ) .  Computational 
difficulties  will  still  arise,  however,  for  the  player  who  does  not 
have  access  to  a  computer.  It  will  be  seen  shortly  that  the 
approximation  developed  in  this  section  is  close  even  for  moderately 
sized  N  . 

The  optimal  last  starting  time,  t*  ,  is  given  in  Theorem 

3.1.  From  corollary  3.1,  it  can  be  seen  that  t *  is  contained  in 

-  1/d 

an  interval  of  length  2  -  e  '  ,  a  value  between  1  and  2.  As 
d  becomes  larger,  the  size  of  this  interval  becomes  closer  to  1. 

An  approximation  to  t*  is  given  by: 


[(N  -  d 


,  .  -1/d 

■*■1)0  +  d 


]  ■ 


(4 


where  [x}  denotes  the  greatest  integer  less  than  cr  equal  to  X. 


The  value  of 


differs  bv  at  most  one  from  the  value 


of 


'd  * 


t^  and 
d 


Most  of  the  time,  especially  as  d  becomes  larger, 

I  » 

will  be  the  same.  The  advantage  to  using  t,  is  its 


computat ional  ease,  especially  when  N  is  at  all  large.  The  Last 
Choice  Rule  can  now  be  defined. 
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Procedure  B  (The  Last  Choice  Rule) :  d  startinq  times  are 
chosen  in  the  following  manner:  the  dth  startinq  time  is  found 
using  (4.1).  By  redefining  N  as  tj-1  ,  and  d  as  d-1  ,  the 
next-to-last  starting  time  is  found  by  aqain  applying  (4.1),  or: 


K  - d 


.11  e-1/ld-U  ♦  d- 


Continue  in  this  manner  until  all  d  startina  times  have  been 
computed.  The  player  then  chooses  objects  on  the  same  basis  as 
Procedure  A. 

The  probabiltiy  of  success  using  the  Last  Choice  Rule  will 
be  denoted  by  Pn(CS)  .  The  Last  Choice  Rule  is  not  optimal,  unless 


t*.  i  =  l,2 . d, 


Tables  1  and  2  aivo  the  starting  times  and  the  probability 
of  success  for  both  the  Optimal  Rule  and  the  Last  Choice  Rule. 

Values  of  d  are  2  and  5,  and  N  -  d(l)  SO ( 10)  100,  1000. 

The  computation  of  the  probability  of  success  when  the 
Last  Choice  Rule  is  used  is  similar  to  the  method  qiven  in  Section  2. 
The  conditional  utilities  of  the  grid  are  given  by: 


i ♦ 1  Hi  ( i ♦ 1 , j )  ♦  j  •  u  ( i ,  j  » 1 ) 

I  ♦3  ♦  i 


if  i-tj  +  l  2  t. 


u  ( i  ,  j  ) 


(4.2) 


i _ •  u  ( i , j ♦ 1 )  ♦  ( j  ♦  1 )  •  u  (  i  ,  j  + 1 )  , 


if  i+j+1  <  t. 


The  difference  between  (4.2)  and  (2.2)  is  that  on  the  optimal 
grid,  given  a  choice,  the  player  chooses  the  point  with  the  larger 
expected  utility.  If  the  starting  times  are  assigned  beforehand, 
as  in  the  Last  Choice  Rule,  the  player  when  given  a  choice,  is  forced 
to  select  an  object  if  the  starting  number  has  been  reached. 

Tables  1  and  2  show  the  accuracy  of  the  approximations.  In 
these  tables,  the  worst  percentage  decrease  usinq  the  Last  Choice 
Rule  for  N  >  10  is  4.2  percent,  when  d=5  and  N=21.  For  d=2, 
as  lonq  as  N  is  greater  than  14  the  percentage  decrease  is  well 
under  1  percent.  The  percentage  decrease  is  under  1  percent  for 
d-3  (not  shown)  when  N  >  20  ,  for  d=4  (not  shown)  when  N  >  26  , 
and  for  d  =  5  when  N  >  32  .  For  N  larqer  than  100,  the  decrease 
is  reduced  to  less  than  .5  percent  in  almost  all  cases. 

While  the  percentage  decrease  in  the  probability  of  success 
using  the  Last  Choice  Rule  is  an  increasing  function  of  d  ,  it 
appears  from  the  calculations  done  so  far  that  this  error  is  still 
very  reasonable.  For  the  statistician  who  does  not  have  access  to 
a  computer,  it  is  an  easy  way  to  approximate  the  optimal  starting 
times  using  only  a  hand-ca ieulator  or  tables  of  the  exponential 
f  unction . 

For  those  with  access  to  a  computer,  programming  algorithm 
(2.2)  is  not  difficult,  and  it  is  a  fast  algorithm  to  run.  This 
would,  of  course,  negate  the  need  for  the  Last  Choice  Rule  as  an 
approximation. 
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TABLE  1 


STARTING  TIMES 

AND  PROBABILITY 

OF  SUCCESS 

FOR  THE 

OPTIMAL 

AND  LAST  CHOICE 

RULES,  d* 

2 

Opt ima 1 

Rule 

Last 

Choice  Rule 

N 

Starting  Times 

(CS) 

A 

PH(CS) 

Starting  Times 

2 

1.2 

1.000000  1 

.000000 

1,2 

3 

1.3 

. 500000 

. 500000 

1,3 

4 

1.3 

.333333 

.333333 

2,4 

5 

2,4 

.333333 

.333333 

2,5 

6 

2,5 

.313889 

. 313889 

2,5 

7 

2,5 

. 295635 

. 295635 

2,5 

8 

2,6 

. 279960 

.279960 

2,6 

9 

3,7 

. 273898 

. 262720 

2,6 

10 

3,7 

.  27  14  33 

.271433 

3,7 

11 

3,8 

. 266299 

.  266299 

3,8 

12 

3,8 

. 260400 

. 260400 

3,8 

13 

4,9 

. 257516 

.254872 

3,9 

14 

4,10 

.255854 

. 248040 

3,9 

15 

4,10 

. 254142 

.254142 

4,10 

16 

4,11 

. 251513 

.251513 

4,11 

17 

5,1 1 

. 248729 

. 248336 

4,11 

18 

5,12 

.  248519 

. 248519 

5,12 

19 

5,13 

. 247338 

. 247067 

5,12 

20 

5,13 

. 246057 

. 246057 

5,13 

21 

5,14 

. 244357 

.244357 

5,14 

22 

6,14 

.  24  3660 

. 242263 

5,14 

23 

6,15 

. 243309 

. 243309 

6,15 

24 

6,  16 

. 242398 

. 242186 

6,15 

25 

6,16 

. 24 1 379 

.241379 

6,16 

26 

7,17 

. 240668 

.240144 

6,17 

27 

7,17 

. 240293 

.238609 

6,17 

28 

7,  18 

.239916 

.239916 

7,18 

29 

7,19 

.  239176 

.239002 

7,18 

30 

8,  19 

.238389 

.238332 

7,19 

31 

8,20 

.238299 

.237370 

7,20 

32 

8,20 

.237901 

.236168 

7,20 

33 

8,21 

.237532 

.237532 

8,21 

34 

8 ,22 

.236909 

.236909 

8,22 

35 

9,22 

.236661 

.236190 

8,22 

36 

9,23 

.236509 

.236509 

9,23 

37 

9,24 

.236133 

.236115 

9,23 

38 

9,24 

.235766 

.235766 

9,24 

39 

10,25 

.235410 

.235228 

9,25 

40 

10,25 

.235292 

.234601 

9,25 

26 


TABLE  1 
(Continued) 


Optimal 

Rule 

Last 

Choice  Rule 

N 

Starting  Times 

(CS) 

A 

PQ(CS) 

Starting  Times 

41 

10,26 

.235111 

.235111 

10,26 

42 

10,27 

.234762 

.234733 

10,26 

43 

10,  27 

.234405 

.234405 

10,27 

44 

11,28 

.  234  34  3 

.233933 

10,28 

45 

11,28 

.  234  184 

.233377 

10,28 

46 

11,29 

.233991 

.233991 

11,29 

47 

11,30 

.233666 

.233632 

11,29 

48 

12,30 

.233539 

.233325 

11,30 

49 

12,31 

.233452 

.233452 

12,31 

50 

12,31 

. 233271 

.233271 

12,31 

60 

14,37 

.231853 

.231853 

14,37 

70 

17,44 

.230943 

.230807 

16,43 

80 

19,50 

.230274 

.230004 

18,49 

90 

21,56 

.  229714 

.  229367 

20,55 

100  . 

24,62 

. 229260 

.229241 

23,62 

1000 

230,608 

. 225816 

.  225739 

223,607 
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TABLE  2 


STARTING  TIMES  AND  THE  PROBABILITY  OF  SUCCESS  FOR  THE 


OPTIMAL 

AND  LAST  CHOICE 

RULES,  d 

=  5 

Optimal 

Rule 

Last 

Choice  Rule 

N 

Starting  Times 

PR (CS) 

A 

P* (CS) 

A 

Starting  Times 

5 

1.2. 3,4.5 

1.000000  1 

. 000000 

1,2, 3, 4, 5 

6 

1. 2,4,5, 6 

.427778 

. 380556 

1,3, 4, 5, 6 

7 

1. 3, 4, 6. 7 

. 295635 

.275397 

1,3, 5, 6, 7 

8 

1 , 3, 5,6,8 

.238194 

.231250 

1,3, 5,7,8 

9 

1. 3, 5, 7, 9 

. 200000 

.200000 

1,3. 5, 7, 9 

10 

2,4,6,8,10 

. 166667 

. 166667 

1,3, 5, 7, 9 

11 

2,4,6,8,10 

. 166667 

. 166667 

2,4,6,8,10 

12 

2,4,6,9,11 

. 161551 

.156249 

2,5,7,9,11 

13 

2,4,7,9,12 

. 156208 

.151789 

2,5,8,10,12 

14 

2,5,7,10,13 

.151445 

. 149512 

2,5,8,11,13 

15 

2,5,8,11,14 

. 147286 

. 147286 

2,5,8,11,14 

16 

2,5,8,11,14 

.142944 

. 142944 

2,5,8,11,14 

17 

2,5,9,12,15 

.138755 

. 1 38676 

2,6,9,12,15 

18 

2, 6, 9, 1 3,  16 

.135380 

. 1 34829 

2,6,10,13,16 

19 

3,6, 10,  13,17 

.132204 

. 131596 

2,6,10,14,17 

20 

3,6,10,14,18 

.131297 

.128442 

2,6,10,14,18 

21 

3,7,11,15,19 

.129967 

. 124524 

2,6,10,14,18 

22 

3,7,11,15,19 

.129375 

.129375 

3,7,11,15,19 

23 

3,7,11,16,20 

. 128149 

. 127480 

3,8,12,16,20 

24 

3,7,12,16,21 

. 126831 

. 126242 

3,8,13,17,21 

25 

3,8,12,17,22 

.125684 

.125346 

3,8,13,18,22 

26 

3,8,13,18,23 

. 1 24446 

.124446 

3,8,13,18,23 

27 

3,8,13,18,23 

. 123047 

.123047 

3,8,13, 18,23 

28 

3,9,14,19,24 

.121769 

.121769 

3,9,14,19,24 

29 

4,9,14,20,25 

.121528 

. 120410 

3,9,15,20,25 

30 

4,9,15,20,26 

. 121159 

.119159 

3,9,15,21,26 

31 

4,9,15,21,27 

. 120655 

. 1 17884 

3,9,15,21,27 

32 

4,10,16,22,28 

.  120110 

. 116301 

3,9,15,21,27 

33 

4,10,16,22,28 

. 119700 

. 119700 

4,10,16,22,28 

34 

4,10,17,23,29 

. 119072 

. 118888 

4,11,17,23,29 

35 

4,11,17,23,20 

.118428 

.1  18246 

4,11,18,25,30 

36 

4,11,17,24,31 

.117841 

.117721 

4,11,18,24,31 

37 

4,11, 18,25,32 

.117188 

.117188 

4,11,18,25,32 

38 

5,11,18,25,32 

. 116839 

.116434 

4,11,18,25,32 

39 

5,12,19,26,33 

.116713 

.116713 

5,12,19,26,33 

40 

5,12,19,27, 34 

. 116508 

. 116453 

5,12,20,27,34 

41 

5,12,20,27,35 

.  116233 

.116194 

5,12,20,28,35 

42 

5,13,20,28,36 

.115891 

.115886 

5,12,20,28,36 

43 

5,13,21,29,37 

. 115570 

.115384 

5,12,20,28,36 

28 


TABLE  2 
(Continued ) 


Optimal 

Rule 

Last 

Choice  Rule 

N 

Starting  Times 

P. (CS) 

A 

Va  (CS) 

n 

Starting  Times 

44 

5,13,21,29, 37 

. 115264 

.115264 

5,13,21,29,37 

45 

5,13,22,30, 38 

.114858 

.114797 

5,14,22,30,38 

46 

5, 14,22,30, 39 

.114466 

.114379 

5,14,23,31,39 

47 

6, 14,22,31,40 

.114241 

.114015 

5,14,23,32,40 

48 

6,14,23,32,41 

.114135 

.113642 

5,14,23,32,41 

49 

6,14,23,32,41 

. 113944 

.113145 

5,14,23,32,41 

50 

6,15,24,33,42 

. 113844 

.113844 

6,15,24,33,42 

60 

7,18,28,39,50 

.112087 

.111178 

6,17,28,39,50 

70 

8,20,33,46,59 

.110912 

.110156 

7,20,33,46,59 

80 

9,23,38,52,67 

.110031 

.110018 

9,23,38,53,67 

90 

10,26,42,59,75 

. 109350 

.  109338 

10,26,32,59,75 

100 

11,29,47,65,83 

.108808 

. 108808 

11,29,47,65,83 

1000 

108,281,460,639, 

820  .104736 

.  104612 

102,278,458,639,1 

29 
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